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Abstract 

Background: Drug-eluting stents (DES) reduce rates of restenosis compared with bare metal stents (BMS). A 
number of observational studies have also found lower rates of mortality and non-fatal myocardial infarction with 
DES compared with BMS, findings not observed in randomized clinical trials. In order to explore reasons for this 
discrepancy, we compared outcomes after percutaneous coronary intervention (PCI) with DES or BMS by multiple 
statistical methods. 

Methods: We compared short-term rates of all-cause mortality and myocardial infarction for patients undergoing 
PCI with DES or BMS using propensity-score adjustment, propensity-score matching, and a stent-era comparison in 
a large, integrated health system between 1998 and 2007. For the propensity-score adjustment and stent era 
comparisons, we used multivariable logistic regression to assess the association of stent type with outcomes. We 
used McNemar's Chi-square test to compare outcomes for propensity-score matching. 

Results: Between 1998 and 2007, 35,438 PCIs with stenting were performed among health plan members (53.9% 
DES and 46.1% BMS). After propensity-score adjustment, DES was associated with significantly lower rates of death 
at 30 days (OR 0.49, 95% CI 0.39 - 0.63, P < 0.001) and one year (OR 0.58, 95% CI 0.49 - 0.68, P < 0.001), and a 
lower rate of myocardial infarction at one year (OR 0.72, 95% CI 0.59 - 0.87, P < 0.001). Thirty day and one year 
mortality were also lower with DES after propensity-score matching. However, a stent era comparison, which 
eliminates potential confounding by indication, showed no difference in death or myocardial infarction for DES 
and BMS, similar to results from randomized trials. 

Conclusions: Although propensity-score methods suggested a mortality benefit with DES, consistent with prior 
observational studies, a stent era comparison failed to support this conclusion. Unobserved factors influencing 
stent selection in observational studies likely account for the observed mortality benefit of DES not seen in 
randomized clinical trials. 



Background 

The comparison of alternative treatments has long been 
a primary mission of both randomized trials and obser- 
vational studies. With the commitment of $1.1 billion in 
support of comparative effectiveness research in the 
American Recovery and Reinvestment Act of 2009, the 
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number of studies comparing different drugs, devices, 
techniques and systems will undoubtedly increase dra- 
matically [1]. While randomized clinical trials are likely 
to remain the gold standard for comparing alternative 
treatments, observational studies should continue to 
have significant, if not leading, roles in comparative 
effectiveness research moving forward, particularly in 
light of recommendations to prioritize assessments of 
community-based interventions within populations tra- 
ditionally underrepresented in clinical trials [2], 
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However, observational studies are subject to a num- 
ber of limitations, foremost among them the potential 
for unmeasured variables that confound results. While a 
number of methods directed at assessing causal effects 
and eliminating confounding have been developed, few 
clinical studies describe reasons for the specific choice 
of method used, and fewer present multiple methods to 
help corroborate findings[3,4]. 

Observational studies comparing drug-eluting stents 
(DES) and bare metal stents (BMS) for percutaneous 
coronary intervention (PCI) have consistently shown 
lower mortality and myocardial infarction associated 
with DES [5-20], findings not seen in randomized clini- 
cal trials [21-27]. We applied three common methods to 
compare DES to BMS within a large observational study 
population to: 1) determine whether mortality benefit 
for DES was observed in our study population and 2) to 
identify potential challenges to the application of these 
methods to compare treatments in the presence of 
strong treatment selection. 

Methods 

We conducted a retrospective dynamic cohort study 
within Kaiser Permanente of Northern California 
(KPNC), a large integrated healthcare delivery system 
caring for > 3.2 million individuals that are broadly 
representative of the local surrounding and statewide 
population [28]. All health plan members aged 30 years 
and older between January 1998 and the end of Decem- 
ber 2007 were considered eligible. The study was 
reviewed by the institutional review board of the Kaiser 
Division of Research, and requirement for informed 
consent was waived due to the nature of the study. 
From this cohort, we identified all PCI procedures using 
either DES or BMS based on relevant International 
Classification of Diseases, Ninth Revision, Clinical Modi- 
fication (ICD-9-CM) and Current Procedural Terminol- 
ogy (CPT) codes that occurred within health plan and 
non-health plan hospitals [29]. Consecutive procedures 
occurring within 7 days of one another were considered 
part of the same clinical episode. Procedures in which 
both DES and BMS were used were excluded from the 
analysis. 

Patient characteristics and coexisting illnesses 

Patient demographic information was obtained from 
health plan electronic databases. Relevant ICD-9 or 
CPT codes found in hospital discharge databases dur- 
ing the eight years before the procedure date were 
used to identify prior cardiovascular disease including 
prior myocardial infarction, prior angina, prior PCI or 
coronary artery bypass graft (CABG) surgery, history 
of ischemic stroke and peripheral arterial disease. Prior 
chronic heart failure was determined based on a 



validated algorithm using information from hospitaliza- 
tion, outpatient and emergency department diagnostic 
codes [30]. 

Relevant data sources were searched for cardiovascu- 
lar risk factors. Diabetes mellitus was identified from a 
validated longitudinal Diabetes Registry relying on 
inpatient and outpatient diabetes diagnoses, receipt of 
anti-diabetic therapies, and abnormal glycosylated 
hemoglobin or blood glucose levels [31]. Hypertension 
was based on serial ambulatory diagnoses or the com- 
bination of diagnoses and receipt of anti-hypertensive 
medications. Chronic lung disease, systemic malig- 
nancy, and history of gastrointestinal bleeding were 
determined based on validated methods [29-35]. The 
presence and severity of chronic kidney disease was 
ascertained using the abbreviated four-variable Modifi- 
cation of Diet in Renal Disease (MDRD) equation for 
estimated glomerular filtration rate based on most 
recent outpatient determination of serum creatinine 
within the previous 24 months before the year exam- 
ined [33]. 

Medication usage 

Prior observational studies comparing outcomes for DES 
and BMS have not included the chronic outpatient use 
of cardioprotective medications, an important potential 
confounder [6,19,36]. Outpatient medication use within 
30 days before PCI was ascertained from health plan 
pharmacy records for therapies known to lower cardio- 
vascular risk, including beta-blockers [37], angiotensin- 
converting enzyme (ACE) inhibitors or angiotensin 
receptor blockers (ARBs), [38] statins [39], and thieno- 
pyridines [40]. Outpatient warfarin use was also 
assessed. More than 90% of patients had a drug benefit 
that provided strong financial incentives to obtain medi- 
cations from health plan pharmacies. 

Primary outcomes 

Death from any cause occurring within thirty days and 
one year after the procedure was identified from health 
plan administrative databases, proxy information, Social 
Security Administration vital status files, and California 
state death certificate informational, 42]. Subsequent 
hospitalized myocardial infarction based on ICD-9-CM 
codes (primary discharge diagnosis coded as 410. xl) 
within one year was also identified. 

Statistical analysis 

Three separate methods were used to compare out- 
comes after DES and BMS: 1) statistical adjustment 
based on propensity score decile, 2) propensity-score 
matching, and 3) a stent era comparison, in which out- 
comes before and after the introduction of DES in 2003 
were compared. 
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Method 1 : propensity-score adjustment 

Because patients were not eligible to receive DES until 
after April of 2003, only PCI procedures after this date 
were included in the propensity score methods. A non- 
parsimonious logistic regression model was used to gen- 
erate propensity scores for the likelihood of receiving 
DES based on 23 demographic and clinical variables. 
Propensity score deciles were created, and logistic 
regression was performed modeling the association of 
DES with outcomes, adjusted for propensity score decile 
(categorical variable) as well as the complete set of 
patient covariates. To accommodate data from indivi- 
duals who may have undergone multiple procedures 
during the study period, robust standard errors account- 
ing for clustering by patient were used. 

Method 2: propensity-score matching 

In the second method, one-to-one nearest neighbor 
matching was performed to compare outcomes after 
DES and BMS in patients undergoing PCI after April 
2003. Caliper width was set at 0.0001 using the STATA 
program psmatch2 (Leuven and Sianesi, 2003) [43]. 
Adequacy of the match was assessed by estimating the 
standardized differences between DES and BMS patients 
for all variables. Outcomes were compared between 



groups using the McNemar's Chi-square test for 
matched comparisons. 

Method 3: stent era comparison 

Because DES was not available before 2003 and from 
2004 onward DES was used in the vast majority of cases, 
a comparison of outcomes of PCI prior to and after DES 
introduction serves as a useful surrogate for assessing the 
impact of DES (Figure 1) [4,36,44]. Procedures between 
January 1998 and March 2003 were assigned to the 'BMS 
era', while those between April 2003 and December 2007 
were assigned to the 'DES era'. This variable would be 
expected to be strongly associated with treatment assign- 
ment, and less prone to unmeasured confounding by 
stent indication. Logistic regression was then performed 
to determine the association of stent era with outcomes, 
adjusted for all assessed covariates. 

Analyses were performed using STATA (Stata Corp. 
2008. Stata Statistical Software: Release 10, College Sta- 
tion, TX: StataCorp LP). A two-sided P value of < 0.05 
was considered significant for all analyses. 

Results 

A total of 35,438 PCIs with stent placement between 
January 1998 and December 2007 were included. Of 



CD 
CD 

I 

c 

o 

</> 

s 

CD 
Q. 

O 

o 
o 

o~ 
o 



CD 

CL 

_CD 
"CD 
DC 



300 



250 - 



200 



150 



100 



50 - 




■Bare Metal Stent 
•Drug-Eluting Stent 



1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 

Year 

Figure 1 Rates of percutaneous coronary intervention with drug-eluting and bare metal stents between 1998-2007, per 100,000 
person-years. All stents prior to April 2003 were bare metal, and 88.3% of stents after April 2003 were drug-eluting. 
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these, 19,103 (53.9%) utilized DES and 16,335 (46.1%) 
utilized BMS. Patients receiving DES were significantly 
different than those receiving BMS in nearly all base- 
line characteristics, including having significantly 
higher rates of hypertensions, diabetes, and congestive 
heart failure (Table 1). DES patients also had lower 



Table 1 Characteristics of Patients Undergoing PCI by 
Stent Type, 1998-2007 





Drug- 
Eluting 
Stent 


Bare 
Metal 
Stent 


P Value 


M 
IN 


1 Q 1 (\t 

i y, i uj 


if: 53C 




Age (years) 


64. 1 


63.2 


< 0.001 


Male Sex (%) 


70.8 


70.6 


0.72 


Race/Ethnicity (%) 






< 0.001 


Asian 


9.8 


7.0 




Black 


5.3 


5.3 




Hispanic 


9.-1 


8.4 




White, non-Hispanic 


fie; Q 






Other 


9.7 


7.3 




Comorbidities (%) 








Diabetes 


33.8 


26.2 


< 0.001 


Hypertension 


68.5 


49.6 


< 0.001 


rilOl IVII 


1 3 n 


1 1 1 


< U.UU I 


PCI Indication: 






< 0.001 


STEMI 


16.0 


27.5 




NSTEMI 


28.0 


19.3 




Stable or unstable angina 


57.3 


54.8 




Prior PCI 


19.1 


14.1 


< 0.001 


Prior CABG 


5.7 


5.3 


0.12 


Congestive Heart Failure 


4.6 


3.4 


< 0.001 


Prior Stroke 


1.6 


1.2 


0.005 


Peripheral Arterial Disease 


4.3 


3.3 


< 0.001 


Glomerular Filtration Rate 






< 0.001 


> 90 


15.0 


16.2 




60 - 89 


39.7 


33.4 




30 - 59 


21.5 


15.2 




15-30 


1.9 


1.3 




< 15 or HD 


0.7 


0.5 




Unknown 


21.1 


33.3 




Systemic Malignancy 


5.7 


5.8 


0.75 


Prior Gastrointestinal Bleeding 


2.3 


1.8 


< 0.001 


Chronic Lung Disease 


18.3 


21.9 


< 0.001 


Medications 








P-Blockers 


50.0 


40.1 


< 0.001 


Clopidogrel 


12.7 


6.3 


< 0.001 


Statin 


51.7 


34.8 


< 0.001 


ACE-I/ARB 


41.9 


27.8 


< 0.001 


Warfarin 


3.8 


3.0 


< 0.001 



Abbreviations: Ml, myocardial infarction; STEMI, ST-elevation myocardial 
infarction; NSTEMI, non-ST-elevation myocardial infarction; PCI, percutaneous 
coronary intervention; CABG, coronary artery bypass graft surgery. 



rates of presentation with ST-elevation myocardial 
infarction within 30 days prior to PCI, and were more 
likely to be receiving outpatient cardioprotective 
medications including statins, beta-blockers and thie- 
nopyridines. Crude-mortality was significantly higher 
at thirty days (2.2% versus 1.6%, P < .001) and one 
year (5.1% versus 4.6%, P = 0.04) after PCI with BMS 
compared to DES. The rate of myocardial infarction 
at one year was 4.7% after BMS and 4.3% after DES 
{P = 0.06). 

Patients receiving BMS after April 2003 were mark- 
edly different than those receiving BMS before April 
2003, having higher rates of hypertension (65.0% versus 
46.8%, P < 0.001), congestive heart failure (4.3% versus 
3.2%, P = 0.008), prior PCI (9.3% versus 5.6%, P < 
0.001), systemic malignancy (9.3% versus 5.2%, P < 
0.001) and prior gastrointestinal bleeding requiring hos- 
pitalization (3.0% versus 1.7%, P < 0.001). Crude-mortal- 
ity at thirty days and one year and myocardial infarction 
at one year after PCI were significantly higher in 
patients receiving BMS after April 2003 compared those 
receiving BMS before 2003 (Figure 2). 

Association between stent type and outcomes by 
analytical method 

After propensity-score adjustment, DES was associated 
with significantly lower mortality at thirty days (OR 
0.49, 95% CI 0.39 - 0.63, P < 0.001) and one year (OR 
0.58, 95% CI 0.49 - 0.68, P < 0.001), as well as a lower 
rate of myocardial infarction at one year (OR 0.72, 95% 
CI 0.59 - 0.87, P < 0.001). After 1:1 propensity-score 
matching (n = 4,126), DES was again associated with 
significantly lower mortality at thirty days (OR 0.55, 95% 
CI 0.35 - 0.85, P = 0.005), and at one year (OR 0.70, 
95% CI 0.52 - 0.92, P = 0.01). Myocardial infarction at 
one year was similar (OR 0.94, 95% CI 0.71 - 1.25, P = 
0.67). All standardized differences between groups were 
< 10%, suggesting an adequate match (Table 2). In the 
stent era comparison, no differences in mortality were 
seen when comparing the DES era versus BMS era at 
thirty days (OR 0.92, 95% CI 0.78 - 1.10, P = 0.39) or 
one year (0.92, 95% CI 0.82 - 1.02, P = 0.12), or in the 
rate of myocardial infarction at one year (OR 0.92, 0.82 
- 1.03, P = 0.16) (Figure 3). 

Discussion 

We have shown that after the introduction of drug-elut- 
ing stents, bare metal stents were reserved for patients 
with significant comorbidities and high mortality rates. 
Commonly used propensity-score methods continued to 
suggest a mortality benefit to DES, while a stent era 
comparison showed no significant differences in all out- 
comes for DES and BMS, similar to results from rando- 
mized clinical trials. 
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Figure 2 Unadjusted outcomes after percutaneous coronary intervention. Outcomes after percutaneous coronary intervention (PCI) with 
drug-eluting versus bare metal stents for the overall cohort are shown in A. Outcomes after PCI with bare metal stents before and after April 
2003 (B). Patients receiving bare metal stents after April 2003 have significantly worse unadjusted outcomes compared to those receiving bare 
metal stents before April 2003. 
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Table 2 Standardized Differences Between BMS and DES 
Patients Before and After Matching, April 1, 2003 - 
December 31, 2007 





Standardized 
Difference Prior to 
Match 


Standardized 
Difference Post 
Match 


N 


(2,518 BMS, 19,101 
DES) 


(2,063 BMS, 2,063 
DES) 


Age (years) 


-8.3 


1.9 


Malp Spy (%) 


-1 7 


-62 


D^r o /Ft h n i ,— ih / 
ndLt:/ LLI 1 MILI Ly 


-1 1 3 


D.O 


Comorbidities (%) 






Diabetes 


9.1 


3.5 


Hypertension 


7.5 


-3.2 


Prior Ml 


2.7 


-1.7 


PCI Indication: 


-21.6 


0.2 


Prior PCI 


6.1 


6.1 


Prior CABG 


2.5 


2.5 


Congestive 
Heart Failure 


3.8 


3.8 


Prior Stroke 


0.0 


0.0 


Peripheral 

r\l Lfcrl Idl LJIitrditr 


1.9 


1.9 


fnlnmpri ilsr 

vJIWI I ICTE UIQI 

Filtration Rate 


0.3 


0.3 


Systemic 
Malignancy 


-13.4 


0.4 


Prior 

Gastrointestinal 
Bleeding 


-4.2 


-0.6 


Chronic Lung 
Disease 


-0.3 


7.8 


Medications 






P-Blockers 


11.9 


-1.1 


Clopidogrel 


8.9 


3.4 


Statin 


12.3 


5.3 


ACE-I/ARB 


9.2 


■11 


Warfarin 


1.0 


3.1 



Abbreviations: Ml, myocardial infarction; PCI, percutaneous coronary 
intervention; CABG, coronary artery bypass graft surgery. 



A number of prior observational studies comparing 
DES to BMS have suggested reductions in mortality 
and myocardial infarction associated with DES, the 
majority of which have utilized propensity-score meth- 
ods to mitigate potential confounding [5-20]. However, 
data from randomized trials have not corroborated 
these findings [21-27]. In a meta-analysis of 22 rando- 
mized trials and 34 observational studies, Kirtane et al 
found no differences in mortality or myocardial infarc- 
tion in randomized trials, but consistent reductions for 
both outcomes in observational studies, and attributed 
these differences, at least in part, to residual confound- 
ing by unmeasured differences between patients receiv- 
ing BMS and DES [23]. Our results are consistent with 



this conclusion. In our cohort, patients who received 
BMS after April 2003 had significantly greater comor- 
bidities and unadjusted death and myocardial infarc- 
tion rates compared to patients who received BMS 
before this date, suggesting that physicians selectively 
reserved BMS for a sicker population of patients after 
DES was introduced. While propensity-score adjust- 
ment and matching both attempt to account for such 
differences, they only account for variables that are 
assessed. In the presence of such a high degree of 
patient selection, any residual confounding in this case 
is likely to lead to the appearance of improved out- 
comes with DES, as was seen in this study. While pro- 
pensity scores have been put forward as a method 
which overcomes some of the limitations of traditional 
logistic regression, they do not address a primary 
threat to the validity of such studies - unmeasured 
confounding. 

A comparison of the BMS era (100% BMS) to the 
DES era (88.3% DES) as a surrogate for a BMS-DES 
comparison eliminates the unwanted influence of 
patient selection [45]. Consistent with this, the stent 
era comparison showed no differences in mortality or 
myocardial infarction between BMS and DES, most 
closely approximating results seen in randomized trials. 
However, because such an analysis compares patients 
undergoing PCI at two different times, important secu- 
lar trends may have influenced the results. For exam- 
ple, comparisons of stent eras in the Medicare 
population and in the New York PCI Registry both 
suggested lower rates of myocardial infarction in the 
DES era compared to BMS era [36,44]. However, 
neither study adjusted for differences in the use of out- 
patient medications such as statins and beta-blockers, 
which have increasingly been used in patients with 
coronary disease over time [46]. In this study, we 
found significantly higher rates of use of beta blockers, 
statins, angiotensin-converting enzyme inhibitors, and 
thienopyridines in the DES era. After accounting for 
these and other secular trends, no differences in myo- 
cardial infarction were observed between DES and 
BMS patients. 

A number of circumstances specific to stent use 
allowed for the critical examination of these analytical 
methods. First, the rapid uptake of DES over BMS cre- 
ated a natural experiment allowing use of a stent era 
comparison. Second, because a large number of patients 
from randomized clinical trials comparing DES with 
BMS have been performed, these studies help provide 
an estimate of the 'truth.' However, in other examples in 
which the questions of interest may be less well studied, 
there could be much greater difficulty interpreting dis- 
parate results. 
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Figure 3 Association between stent type and 30-day and 1-year mortality, and myocardial infarction at 1 year using different 
analystical methods. Results are for (A) propensity-score adjustment; (B) 1:1 propensity score matching; and (C) a stent era comparison. The 
stent era comparison gives results most consistent with findings from randomized clinical trials. 



Conclusions suggested strong treatment selection, which were 

In summary, we have compared outcomes after PCI incompletely adjusted for by propensity-score meth- 

with either DES or BMS, using three commonly used ods. In the presence of such treatment selection, 

methods of adjustment in observational studies. Large alternative methods of analysis of observational data 

baseline differences in BMS and DES groups should be considered. 
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